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1  Purpose/Scope 

As  projects  become  increasingly  complex,  the  need  for  good  systems  engineering  is  paramount. 
Part  of  systems  engineering  is  documentation  of  past  events,  current  activities,  and  future  plans 
of  software,  hardware,  and  technical  management.  Documentation  of  the  program’s  technical 
plans  can  require  significant  effort  to  keep  current  and  to  keep  the  content  synchronized  in  an 
environment  where  change  is  constant.  Add  to  this  issue  the  fact  that  many  documents  have 
overlapping  information  and  you  have  additional  complications.  This  often  results  in  the 
documents  becoming  obsolete  relative  to  fast  moving  development  activities  and  can  create 
inconsistencies  across  the  project.  The  objective  of  this  research  was  to  investigate  the  ability  to 
align  systems  engineering  (SE)  documents  such  that  the  program  documents  track  and 
complement  one  another,  are  easier  to  produce  and  update,  support  agile  environments,  and 
move  towards  a  data  centric  rather  than  document  centric  focus.  The  documents  that  were 
evaluated  for  this  task  were  the  Systems  Engineering  Plan  (SEP),  Test  and  Evaluation  Master 
Plan  (TEMP),  and  the  Information  Support  Plan  (ISP).  The  tool  utilized  to  assist  in  this  research 
work  was  the  Systems  Engineering  Toolkit  (SET)  developed  by  the  Rotorcraft  Systems 
Engineering  and  Simulation  Center  (RSESC)  at  the  University  of  Alabama  -  Huntsville. 

2  Applicable  Documents 

The  following  were  the  documents  or  guidance  that  were  used  or  referenced  as  a  part  of  this 
research: 

•  Department  of  Defense  Instruction  (DODI)  Number  4630.8,  Procedures  for 
Interoperability  and  Supportability  of  Information  Technology  (IT)  and  National  Security 
Systems  (NSS),  June  30,  2004. 

•  Defense  Acquisition  Guidebook  (DAG),  Traditional  Information  Support  Plan,  Defense 
Acquisition  University. 

•  Department  of  Defense  (DoD)  Enhanced  Information  Support  Plan  (EISP)  Guidebook 
Version  2.0,  November  16,  2009. 

•  Department  of  Defense  Systems  Engineering  Plan  Preparation  Guide  Version  2.01,  April 
2008. 

•  Defense  Acquisition  Guidebook  Annex,  Test  and  Evaluation  Master  Plan. 

3  SET  Tool  Background 

The  SET  was  originally  created  to  assist  in  the  development  of  SEPs.  The  hope  was  to  create  a 
tool  that  assisted  programs  in  building  their  technical  planning  documents  so  that  more  time 
could  be  spent  planning  verses  creating  a  document.  The  tool  takes  the  administrative  work  such 
as  the  creation  of  table  of  contents,  formatting,  building  of  acronym  lists,  and  the  numbering 
tables  and  figures  out  of  the  writers’  hands  thus  allowing  concentration  on  the  information 
required  by  the  document.  The  beta  version  of  the  tool  was  released  in  June  of  2007  with 
version  1.0  being  released  in  March  of  2008.  An  updated  version  is  anticipated  in  early  2011. 


1 


Additional  research  is  ongoing  to  further  develop  the  tool  and  capabilities  to  create  a  family  of 
systems  engineering  documents  with  funding  from  NAVAIR,  DoD,  and  NASA  Marshall  Space 
Flight  Center. 


3.1  SET  Tool  Overview 

The  University  of  Alabama  in  Huntsville/Rotorcraft  Systems  Engineering  and  Simulation  Center 
developed  a  web-based  systems  engineering  plan  generation  tool.  It  is  part  of  an  overall  systems 
engineering  toolkit.  This  tool  was  designed  to  assist  a  program’s  technical  planning.  The  output 
product  is  a  systems  engineering  plan  (SEP)  which  documents  the  systems  engineering  processes 
of  a  program  per  guidelines  established  by  the  Office  of  the  Under  Secretary  of  Defense.  The 
SET  allows  technical  experts  to  document  the  systems  engineering  tasks  in  their  area  of  expertise 
simultaneously  thus  increasing  the  overall  productivity  and  effectiveness  of  the  planning  process. 

The  tool  is  a  web-based  systems  tool  with  data  fields  that  present  content  pertinent  to  the  type  of 
program  and  the  phase  of  the  program  generated  from  a  database.  Tight  configuration 
management  is  maintained  by  the  fact  that  data  may  only  be  modified  from  within  the  web-based 
tool  and  all  changes  to  data  fields  are  captured  in  a  change  log  associated  with  the  project.  SET 
does  not  require  any  installations  on  the  user’s  computer  thus  allowing  access  in  many 
environments.  The  SET  compiles  user  input  into  systems  engineering  documents  such  as  the 
SEP.  These  outputs  are  presented  as  compiled  PDF  documents  that  may  be  reviewed  by  the 
project  team  members,  upper  management  or  the  appropriate  approving  authority  outside  the 
SET  to  support  existing  document  review  processes.  The  SET  also  provides  an  integrated 
review  process  that  to  track  all  comments  submitted  and  to  allow  reviewers  to  know  the  status  of 
the  plan  at  any  point.  The  tool  automatically  generates  the  table  of  contents  and  many  of  the 
tables  within  the  document  that  can  be  time  consuming  for  the  writers. 

The  web-based  feature  enables  different  personnel  to  work  on  the  document  in  their  specialized 
areas  without  worrying  about  latest  versions,  formatting  issues,  configuration  management  or 
having  the  bottleneck  of  one  sole  figure  that  is  responsible  for  incorporating  all  the  sections  and 
variations.  Avoiding  this  bottleneck  allows  the  document  preparation  process  to  be  more 
efficient.  The  modularity  of  the  tool  allows  it  to  be  tailored  to  many  types  of  documents  and 
projects.  Multiple  users  may  access  the  tool  to  write  or  review  the  documents  at  the  same  time. 
The  managers  and  reviewers  may  also  know  at  any  point  the  status  of  the  document  by  utilizing 
the  status  indicators. 

The  tool  enhances  the  communication  process  which  is  a  crucial  role  in  systems  engineering. 
Messages  may  be  sent  from  within  the  tool  and  are  displayed  within  and  outside  of  the  tool  for 
writers  in  preparation  of  the  document  as  well  as  reviewers  and  approvers  in  overseeing  the 
document.  Another  communication  enhancement  is  the  capability  to  log  into  the  SEP  at  any 
time  to  see  the  present  status  and  items  that  have  been  modified  within  the  documents. 
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Some  of  the  key  features  of  the  web-based  tool  are  as  follows: 

1 .  Configuration  management  with  global  access 

2.  Multiple  users  on  different  sections  at  any  given  time 

3.  Secured  and  controlled  access  (account  required) 

4.  Modular  based  software  designed  to  meet  different  program  office’s  needs 

5.  Internal  mapping  capabilities  to  assist  users  when  moving  from  milestone  to  milestone  or 
if  there  are  updates  to  the  guidance 

6.  Integrated  creation  and  review  process  available  to  speed  the  planning  process  and 
documentation  creation  time 

7.  Can  create  consistency  across  numerous  documents 

8.  Integrated  communications  tools 

9.  Foundation  for  metrics  and  statistical  analysis 

10.  Ability  to  upload  images  and  appendices 

11.  Up  to  date  dynamic  knowledge  on  the  status  of  the  program’s  systems  engineering 
documentation  by  any  of  the  teammates 

12.  Ability  to  cross  program  lines  when  dealing  with  systems  of  systems  or  family  of  systems 
SEPs 

13.  Ability  to  create  multiple  types  of  documents  containing  the  same  or  dependent  content 
using  SET’s  mapping  capabilities  to  increase  consistency  and  efficiency 

14.  Preset  tailoring  of  the  SEP  depending  on  program  AC  AT  level  or  program  phase 

15.  Tailorable  tool 

16.  Conformity  to  the  guide  therefore  creating  a  simpler  review  process 

17.  Ensures  the  entire  lifecycle  is  addressed  therefore  it  can  be  more  thorough 

18.  Instant  feedback  to  the  writers  and  reviewers 

19.  Reference  and  example  data  fields 

Future  enhancements  to  the  toolkit  include  research  into  linkages  to  other  programmatic  and 
acquisition  type  documents.  This  may  include  such  things  as  acquisition  strategies,  Test  and 
Evaluation  Master  Plans,  Interface  Control  Documents,  Capability  Production  Documents,  and 
team  charters,  eliminating  redundant  efforts  and  ensuring  that  the  information  in  the  documents 
agree  and  are  consistent  with  one  another. 

Other  linkages  may  be  created  with  the  Systems  Engineering  Management  Plan  (SEMP)  that  is 
developed  by  the  prime  contractor  and  possibly  the  work  breakdown  structure  (WBS)  and 
request  for  proposal  (RFP)  generation. 

As  seen  from  this  research  there  are  many  ways  for  growth  in  making  the  technical  planning  for 
a  program  more  efficient.  It  also  begins  to  open  the  door  for  developing  complex  systems,  SoS 
and  FoS  programs.  This  is  one  of  the  many  ways  to  improve  the  overall  aspects  of  the  technical 
planning  of  a  program  and  move  the  documentation  process  along  into  the  generation  of 
platforms. 
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Figures  3-1  to  3-5  are  screen  captures  of  the  user  interface  and  various  features  that  the  SET 
offers.  Further  details  on  the  tool  can  be  found  in  the  SET  Quick  Start  Guide  located  in  the 
appendix  of  this  document. 
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Figure  3-1.  SET  Navigation  Screen 
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0b  Provide  a  program  schedule  which  shows  major  milestones.  SE  technical  reviews,  and  notional  dates  for  major  events 
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b.  Given  the  Requirements  outlined  in  System  Capabilities.  Requirements,  and  Design  Considerations,  who  are  the 
appropriate  technical  authorities? 
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system  LSE  and  coordinates  with  die  Navy  counterpart  LSE.  The  LSE's  primary  responsibility  is  the  day-to-day  application  of  systems 
engineering  principles,  processes,  and  products  and  coordinates  with  the  SE  Director,  the  JAGM  system  Product  Manager,  and  JAMS 
Project  Manager  (PM)  through  joint  IPT  process 
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Figure  3-5.  Generation  of  Draft  or  Working  Copy  of  Report 


SET  provides  eight  types  of  users  allowing  customized  document  generation  and  review  process 
that  works  for  your  organization.  Users  may  be  assigned  multiple  roles  to  allow  greater 
flexibility  within  the  tool.  Available  user  roles  are  as  follows: 

•  Reader  -  Lowest  level  of  permissions,  only  able  to  generate  document 

•  Writer  -  User  populates  the  document 

•  Reviewer  -  Reviews  the  document  at  an  inquiry  level 

•  Peer  Reviewer  -  Reviews  the  document  at  an  inquiry  level  (Note:  Peer  roles  do  not 
effect  document  processing,  inputs  are  merely  advise). 

•  Approver  -  Approves  the  document  at  the  section  level 

•  Peer  ApproverError!  Bookmark  not  def,ned'  -  Approves  the  document  at  the  section  level 

•  Version  Controller  -  Final  approver  of  the  document,  one  person 

•  Administrator  -  Sets  up  user  roles,  document  type,  etc. 


An  illustration  of  the  document  development  and  internal  review  process  capabilities  can  be 
found  in  the  Figure  3-6. 
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An  additional  feature  of  the  tool  that  is  hidden  to  users  is  the  mapping  and  tailoring  aspect.  As 
guidance  is  updated,  the  tool’s  internal  mapping  process  can  update  a  user’s  document  to  a  newer 
set  of  guidelines  automatically  to  ensure  that  the  program’s  documents  reflect  the  latest  guidance 
or  policy.  Figure  3-7  is  an  illustration  of  how  complex  this  process  can  be  for  a  user.  By  having 
the  tool  automatically  update  the  documents,  it  saves  time  for  the  planners  such  that  their  time  is 
used  for  planning  verse  modifying  a  document  or  trying  to  figure  out  the  changes.  The  tool 
maintains  the  older  versions  of  guidance  at  all  times  in  cases  where  an  organization  is  not  ready 
to  move  to  the  new  guidance  or  is  not  required  to  update  the  document.  This  mapping  capability 
is  also  used  as  a  program  progresses  through  the  lifecycle,  when  a  program  moves  between 
milestones,  the  contents  within  the  documents  are  automatically  updated  to  reflect  the  next 
Milestone,  and  pertinent  text  is  flowed  forward  as  shown  in  Figure  3-8  when  desired. 
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Guide  VI. 02 


Guide  V  2.1  +  Addendum 


1.0  Introduction 

1.1  Program  Description  and  Applicable  Documents 

1.2  Current  Program  Status 

1.3  Approach  for  SEP  Updates 
2.0  Program  Requirements 

2.1  Capabilities  and  Key  Performance  Parameters 

2.2  Statutory  and  Regulatory  Requirements 

2.3  Specified  and  Derived  Requirements 

2.4  Certification  Requirements 

2.5  Design  Considerations 

2.6  Individual  Design  Considerations 

3.0  Technical  Staffing  and  Organizational  Planning 

3.1  Lead/Chief  Systems  Engineer  and  Functional  Leads 

3.2  IPT  Organization/Structure 

3.3  IPT  Staffing/Functional  Skills 

3.4  IPT  Coordination 

3.5  Integration  with  Contractors  and  External  Organizations 
4.0  Technical  Baseline  Management 

4.1  Technical  Baseline  Management  Responsibility 

4.2  Defining,  Approving  and  Maintaining  the  Technical  Baseline 

4.3  Requirements  Traceability  and  Verification  and  Validation 

4.4  Specification  Tree  and  WBS  Link 

4.5  Technical  Maturity 

5.0  Technical  Review  Planning 

5.1  Event-Driven  Technical  Reviews 

5.2  Technical  Review  Management 

5.3  Chairing  of  Technical  Reviews 

5.4  Stakeholder  Participation  in  Technical  Reviews 

5.5  Peer  Participation  at  Technical  Reviews 

6.0  Integration  with  Overall  Management  of  the  Program 

6.1  Linkage  to  Other  Program  Management  Plans 

6.2  Program  Manager's  Approach  to  Using  Technical  Reviews 

6.3  Risk  Management  Integration 

6.4  Test  and  Evaluation 

6.5  Sustainment  Integration 

6.6  Contracting  Considerations 


1.0  Introduction 

1.1  Program  Description  and  Applicable  Documents 

1.2  Program  Technical  Status  at  Time  of  SEP  Submittal  v  | 

1.3  Approach  for  Updating  SEP  — 
2.0  Systems  Engineering  Application  to  Life  Cycle  Phases 

2.1  System  Capabilities,  Requirements,  and  Design  Considerations 

2.1.1  Capabilities  to  be  Achieved 

2.1.2  Key  Performance  Parameters 

2.1.2.1  Individual  KPPs 

2.1.3  Statutory  and  Regulatory  Requirements 

2.1.4  Certification  Requirements 

2.1.5  Design  Considerations 

2.1.4.1  Design  Constraint  Subsections 

2.2  Systems  Engineering  Organizational  Structure 

2.2.1  Organization  of  IPTs 

2.2. 1.1  Individual  IPTs 

2.2.2  Organizational  Responsibilities 

2.2.2.1  Additional  Subsections 
2.23  Integration  of  SE  into  Program  IPT’s 

2.2.4  Technical  Staffing  Plan 
2.3  Systems  Engineering  Process' 

2.3.1  Process  Selection 

2.3.2  Process  Improvement 

2.3.3  Tools  and  Resources  ^ 

2.3. 3.1  Test  and  Evaluation 

2. 3.3.2  Modeling  and  Simulation 

2. 3.3.3  Measures  of  Effectiveness 

2.3.4  Approach  for  Trades  '  jA 

2.4  Technical  Management  and  Control  ^ 

2.4.1  Technical  Baseline  Management  and  Control  (Strategy  and  Approach) 

2.4.2  Technical  Review  Plan  (Strategy  and  Approach) 

2.4.2.1  Technical  Reviews 

2.5  Integration  with  Other  Program  Management  Control  Efforts 

2.5.1  Acquisition  Strategy' 

2.5.2  Risk  Management 

2.5.3  Integrated  Master  Plan 

2.5.4  Earned  Value  Management  -  -  j 

2.5.5  Contract  Management  ~ 


Figure  3-7.  Guidance  Updates 


MSA 


MS  B 


MS  C 


Figure  3-8.  Milestone  Mapping 
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4  Research  Methodology 

The  following  definitions  will  assist  in  understanding  the  research  methodology: 

•  Correlated  Information  -  Duplicate  topic  information  found  in  more  than  one  document 
with  only  one  governing  entity.  A  governing  entity  can  be  a  document  or  role  filled  by 
personnel  as  clarified  below. 

-  Governing  Document  -  Topic  areas  are  dependent  on  specific  documents  such  as  the 
SEP,  TEMP,  ISP,  etc.,  not  necessarily  a  particular  role  or  SME.  The  governing 
document  controls  the  content  and  changes  to  that  content  for  a  subject  area. 

(Generic  roles:  reader,  writer,  reviewer,  approver,  version  controller) 

-  Governing  Role  -  Independent  topic  areas  and  not  governed  by  a  specific  document. 
This  information  would  be  changed  by  preapproved  individual  roles.  Changes  to  the 
information  are  not  governed  by  the  document.  (Specific  Roles:  PM,  LSE,  SMEs, 
Logisticians,  etc.) 

•  Dependent  Information 

-  Level  1 :  High  level  details  about  a  topic  area.  An  overview  on  how  processes  will  be 
handled.  Should  be  consistent  with  Level  2  information. 

-  Level  2:  Lower  level  more  specific  information  that  falls  in  line  with  the  Level  1 
information  but  has  much  more  detail  specifics. 

Three  documents  were  identified  for  this  research:  SEP,  TEMP,  and  ISP.  The  first  step  was  to 
understand  the  information  that  each  document  required  by  creating  a  modular  architecture  for 
each  one.  Prom  this,  high  level  topic  areas  were  generated  for  each  document  from  the  guidance 
or  policy.  Then  a  commonality  matrix  was  created  between  the  topic  areas  to  understand 
common  themes.  Further  analysis  was  then  performed  on  the  high  level  topic  areas  to  determine 
whether  the  topic  area  information  was  dependent  or  correlated  information.  The  next  step  was 
to  classify  the  dependent  information  as  Level  1  or  Level  2. 

A  commonality  table  of  contents  was  generated  that  merged  these  modular  architectures  where 
possible  in  order  to  investigate  the  linkages  and  dependencies  across  the  documentation.  Each 
document  topic  area  was  mapped  to  the  new  table  of  contents  noting  what  sections  or  topic  areas 
were  not  mapped.  The  topics  in  the  commonality  table  of  contents  assisted  in  determining  the 
governing  document  as  well  as  the  governing  role.  Based  on  the  governing  document,  the 
dependent  information  was  evaluated  as  to  which  document  contained  the  level  1  information 
while  which  document  had  the  level  2  information.  Once  completed,  a  methodology  for  utilizing 
the  results  in  the  most  effective  manner  was  developed  to  allow  better  change  management, 
maintain  consistency,  and  leverage  the  modularity.  The  final  product  was  built  on  existing 
capabilities  of  the  SET  developed  by  RSESC. 
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5  Results/Discussion 


The  research  began  by  reviewing  the  guidance  for  each  of  three  key  SE  documents:  SEP, 

TEMP,  and  ISP  (as  listed  under  applicable  documents).  Based  on  the  topics  presented  in  each  of 
the  documents,  a  new  table  of  contents  (commonality  table  of  contents)  was  developed  that 
encompassed  areas  of  overlap  between  2  or  more  documents.  The  analysis  was  performed  from 
two  complimentary  directions.  One  way  in  which  the  data  was  investigated  was  by  looking  at 
the  topic  areas  noting  the  amount  each  document  that  overlapped  with  another  document.  The 
second  approach  was  to  assess  each  document’s  table  of  contents  how  they  mapped  into  the  topic 
areas.  An  example  of  a  topic  area  would  be  Mission  Need  or  Previous  Testing.  In  regards  to  the 
Mission  Need,  it  was  identified  as  a  topic  area  located  in  all  three  documents  while  Previous 
Testing  was  common  to  only  the  TEMP  and  SEP.  The  final  analysis  showed  the  section  titles, 
topic  areas  and  mappings  of  the  documents  to  different  topic  areas.  An  example  is  presented  in 
Table  5-1.  The  results  were  that  a  large  portion  of  each  document  mapped  into  the  commonality 
table  of  contents.  There  were  seventy-six  topic  areas  that  were  in  common  between  at  least  two 
of  the  three  documents.  For  example,  the  SEP  had  fifty-two  topic  areas  that  were  in  common 
with  another  document  or  sixty-eight  percent  of  the  document  was  in  common  with  another 
document.  Note:  Commonality  means  the  information  was  either  correlated  information  or 
dependent  information.  A  break-out  of  each  document  is  presented  in  Table  5-2.  When 
examining  the  table  of  contents  for  each  of  the  three  documents,  greater  than  55%  had 
commonality  as  presented  in  Table  5-3.  Number  of  orphan  sections  refers  to  the  sections  that 
were  specific  to  a  document.  For  instance,  live  fire  test  and  evaluation  approach  or  TEMP 
updates  is  only  discussed  in  the  TEMP,  therefore  these  sections  would  be  considered  orphans. 
The  results  for  the  ISP  were  more  difficult  to  evaluate  because  there  was  more  than  one 
document  that  guides  the  development  of  an  ISP.  A  rough  table  of  contents  was  developed 
based  on  the  guidance  and  an  example  ISP  that  was  provided. 

Table  5-1.  Mapping  Results 


Topic  Areas 

Level 

Governing  Entity 

TEMP  Section 

SEP  Section 

Milestone 

ISP 

(DO  DI/DAG  1 

ISP  Example 

Mission  Need 

1 

Role  Based/SME 

1.2 

2 

A.  R  and  C 

2-1 

2.1 

Supported  Capability 

2 

Role  Based/SME 

2.2 

2.2 

OV-1  Showing  the  operational  environment 

1 

Role  Based/SME 

1.2 

1.1 

A.  B,  and  C 

1.1 

1.1 

Organizations  which  the  system  will  be  integrated  (if  applicable 

1 

Role  Based/SME 

1.2 

3.5 

A.  8.  and  C 

1.1 

1.1.1 

Role  Definitions 

2 

Role  Based/SME 

1.2 

1.3 

Business  Case 

1 

Role  Based/SME 

1.2 

1.1 

A,  B,  and  C 

System  Description  and  Configuration 

1 

Role  Based/SME 

1.1 

1.1 

A. B. and  C 

1.1 

1.1 

Key  features 

2 

Role  Based/SME 

1.3 

1.1 

A.  B.  and  C 

1.1 

1.1 

Required  Capabilities 

2 

Role  Based/SME 

2.4 

2.4 

Threat  Environment 

1 

Role  Based/SME 

1.1.1 

1-1 

A.  8.  and  C 

11 

1.1 

Analysis  of  Alternatives 

1 

Role  Based/SME 

1.1.2 

4.4 

A 

Appendix  A 
refers  to  it 

Touches  on  this  In  i  l  l 
and  1.3.2.1  but  no  big 
discussion 
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Table  5-2.  Commonality  Mapping 


Document 

Topic  Areas  with 
Commonality 

Percent 

Commonality 

SEP 

52 

68% 

TEMP 

49 

64% 

ISP  (DODI/DAG) 

21 

28% 

ISP  (Example) 

24 

32% 

Table  5-3.  Table  of  Contents  Mapping 


Document 

Total 

Number  of 

Sections 

Number  of 
Orphan  Sections 

Number  of  Sections 

with  Common 

Information 

Percent 

Common 

SEP  MS  A 

29 

10 

19 

65.5% 

SEP  MS  B 

29 

11 

18 

62.1% 

SEP  MS  C 

29 

13 

16 

55.2% 

TEMP 

57 

26 

14 

56.1% 

ISP  (DODI/DAG) 

23 

9 

14 

60.8% 

Figure  5-1  is  an  illustration  of  the  mapping  between  the  TEMP  and  SEP.  This  is  an  example 
just  between  two  but  it  was  completed  on  all  three  documents. 
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Figure  5-1:  Document  Mapping  Example 

As  this  mapping  progressed,  a  preliminary  determination  was  made  to  answer  such  questions  as 
which  document  governs  this  information,  is  this  information  governed  by  a  role  such  as  project 
manager,  or  how  detailed  does  your  document  need  to  be  in  relation  to  other  documents  that 
discuss  the  same  information.  A  sample  of  the  results  is  presented  in  Table  5-1. 

As  part  of  the  research  task  a  project  based  system  was  developed  for  managing  the  data  as 
discussed  for  the  three  documents.  A  semantic  data  model  was  developed  that  allowed  the 
creation  of  project  schemas  in  terms  of  concepts  and  their  relationships.  After  the  creation  of  a 
schema,  the  system  allows  for  the  creation  of  a  project  that  conforms  to  a  particular  project 
schema.  In  order  to  map  the  data  into  a  form  that  matches  the  format  of  the  SEP,  TEMP,  and 
ISP,  a  modular,  node-based  presentation  system  was  developed  for  specifying  transformations  of 
projects  into  a  document  like  format.  A  role-based  permission  system  was  implemented  that 
allowed  for  tailorable  sets  of  permissions  that  could  be  selected  based  on  the  user's 
organizational  needs.  Permission  assignment  was  implemented  at  both  the  project  schema  level 
and  the  presentation  node  level.  A  web-based  AJAX  user  interface  was  developed  that  allowed 
the  user  to  edit  the  project's  underlying  data  using  the  document-like  format  produced  by  the 
presentation  system. 

As  the  team  began  to  discuss  these  results,  questions  arose  about  how  to  make  the  modular 
database  function  effectively.  Migrating  to  a  modular  database  could  increase  efficiency, 
synchronization,  and  consistency  across  multiple  documents.  How  can  a  program  become  data 
centric  rather  than  document  centric?  The  answer  that  came  out  was  to  be  more  role  based. 
Evidence  showed  various  subject  matter  experts  are  needed  within  a  project  and  can  vary 
between  milestones  (chief  engineer,  lead  system  engineer,  project  manager,  test  lead, 
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logisticians,  etc.).  Research  on  these  three  documents  showed  topic  areas  that  SMEs  govern  and 
are  co-located  in  multiple  documents.  The  method  for  utilizing  this  information  is  to  develop  a 
set  of  topics  that  are  specific  to  each  SME.  Principal  writers  or  SMEs  are  selected  for  the 
predetermined  topic  areas.  Each  SME  would  access  the  SET  and  be  able  to  select  a  meta 
document  which  is  tailored  to  their  area  of  expertise.  This  governing  information  is  then  made 
available  to  the  pertinent  documents  that  have  been  mapped  to  the  information.  (This 
information  could  be  pulled  from  already  written  documents  within  the  tool,  require  newly 
developed  information  or  a  combination  of  the  two.).  An  example  is  presented  in  Figure  5-2. 

The  results  would  be  that  if  a  user  were  to  access  the  tool  to  write  a  TEMP,  they  would  pull  the 
TEMP  template  and  any  information  that  has  already  been  written  by  the  principal  writers  or 
SMEs  would  already  be  populated  into  the  document.  The  only  topics  to  be  answered  are  the 
areas  that  are  specific  to  that  particular  document.  A  schematic  is  presented  in  Figure  5-3.  Some 
of  the  benefits  from  this  methodology  are: 

•  Reduces  the  time  necessary  to  complete  a  document 

•  Ensures  that  common  information  between  documents  is  consistent 

•  As  information  is  updated  by  principals  or  SMEs,  all  documents  are  updated 

•  The  review  process  could  go  faster  because  much  of  the  information  has  been  supplied  by 

principals  or  SMEs  who  are  often  also  a  reviewer  to  large  sections  of  the  document 

•  Documents  can  be  frozen  and  version  controlled  at  each  milestone 


Home  Messages  Options  View  ▼  C 

©  Documents 
©  e  Demo  Project 

Project  Copfiatf  a  bon 

S££ 

TEMP 
ISP 

Proiect Info 

Attachments 
Q  Demo  Project  2 

Proiect  Configuration 

SEP 
TEMP 

ISP 

Proieyt Info 

Attachments 


Home  Messages  Options  View  ▼  Change  Password  Logout 

B  /  U  Link  nhi'ElEOtlfll11!  x 


^  Zt  S'  ffl  VD  'a  O' 


a.  Mission  Need 

Due  to  a  lack  of  situational  support  in  the  theater  of  operations  it  is  apparent  that  a  new  capability  to  provide  both  support  and  diagnost 

b.  Mission  Need  -  Supported  Capability 
The  supported  capability  will  provide  awareness,  diagnostics  and  prevention  of  existing  problems  while  building  on  existing  capabilities 

c.  OV-1  Showing  the  operational  environment 
[Diagram  Here] 

d  Organizations  which  the  system  will  be  integrated  (if  applicable) 

For  this  effort  we  will  coordinate  with  the  applicable  organizations  in  order  to  leverage  resources  and  ensure  stakeholder  interests  are  r 

e  Organizations  which  the  system  will  be  integrated  -  Role  Definitions 
The  developers  will  develop  a  solution  and  will  coordinate  with  manufacturers  and  operations  personnel  to  ensure  proper  integration  of 

f.  Business  Case 

Due  to  the  current  costs  associated  with  the  existing  lack  of  situational  support,  and  the  current  support  for  this  solution  by  our  partner  o 
sufficient  need  and  funding  to  proceed  with  this  project 

g  System  Description  and  Configuration 
The  system  will  be  composed  of  modular  components  which  will  integrate  to  meet  mission  requirements  These  components  will  be  co 
configuration  system  which  both  provides  ease  of  configuration  as  well  as  ensures  optimal  configuration 

h.  System  Description  and  Configuration  -  Key  features 

The  systems  key  features  will  include  it's  ease  of  use  and  ability  to  support  optimal  configurations. 

i.  Threat  Environment 

The  threat  environment  will  be  complex  and  will  be  determined  primanly  by  the  theater  of  operations 

j  Analysis  of  Alternatives 
Contact  the  A  team. 

k.  Acquisition  Strategy  Overview 


Figure  5-2.  Example  SME  Information  Request 
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Figure  5-3.  Role  Based  Data  Entry 

There  are  disadvantages  to  this  methodology  in  that  different  people  have  different  writing 
styles.  This  can  be  very  evident  when  evaluating  the  completed  document.  Also,  it  can  make  the 
final  document  not  read  as  smoothly  as  when  a  single  person  is  the  writer.  Both  of  these  issues 
could  be  handled  by  using  a  technical  writer  that  could  assist  in  creating  a  more  readable 
document.  Precautions  would  need  to  be  taken  so  the  technical  writer  did  not  alter  core 
information  supplied  by  the  principals/SMEs.  Also,  it  would  take  a  little  time  to  get  all  users 
involved  comfortable  in  their  roles  and  understanding  the  process.  The  thought  is  that  the 
advantages  outweigh  these  issues. 

6  Conclusions 

The  research  presented  showed  that  using  a  data-centric  modular  database  for  creating  program 
documentation  is  feasible  and  could  be  beneficial.  Evidence  was  found  that  there  are 
dependencies  and  correlations  between  the  three  documents.  The  automated  mapping  function, 
database  capabilities,  statistical  and  data  collection  methods  designed  within  the  SET  tool  allow 
both  role  based  and  document  based  planning.  It  also  provides  both  a  testbed  environment  and 
implementation  tool  for  users. 

7  Future  work 

Based  on  these  results,  there  are  several  topics  for  future.  Determining  a  higher  fidelity  of  the 
topic  areas  and  information  requests  would  be  one  area  while  further  defining  and  finalizing  the 
level  1  and  2  mappings.  Advancing  the  documentation  process  as  well  as  determining  roles  and 
governing  entities  would  take  this  research  to  the  next  level. 
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Appendix  A:  Topic  Areas 


1.  Mission  Need 

a.  Support  Capability 

2.  OV-1  showing  the  operational  environment 

3.  Organizations  which  the  system  will  be  integrated  (if  applicable) 

a.  Role  Definitions 

4.  Business  Case 

5.  System  Description  and  Configuration 

a.  Key  features 

b.  Required  capabilities 

6.  Threat  Environment 

7.  Analysis  of  Alternatives 

8.  Acquisition  Strategy  Overview 

9.  Previous  Testing 

10.  KPPs,  KSAs 

1 1 .  Key  Interfaces 

a.  System  interfaces 

b.  External  interfaces 

c.  Risk  interfaces 

d.  Interface  control  agreements 

12.  System  Architectures  that  are  required  for  mission  accomplishment 

a.  Inconsistencies 

13.  Certification  Requirements 

14.  Unique  System  Characteristics 

a.  Special  tests 

15.  SE  Requirements 

a.  Relative  to  Test  and  Evaluation 

16.  Test  and  Evaluation  Responsibilities 

a.  Initial  Operational  Test  and  Evaluation 

17.  IPTs  and  WIPTs 

a.  Technical  authority 

b.  Structure 

c.  Staffing 

d.  Coordination 

e.  Integration  with  contractor  and  external  organizations 

f.  ISE  IPT  details  if  there  is  an  ISP  IPT  or  WIPT 

18.  Modeling  and  Simulation 

a.  SEP  Specific 

i.  Tools 

ii.  SE 
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iii.  Analysis 

iv.  V&V 

v.  Requirements 

vi.  T&E 

vii.  Life  cycle 

b.  TEMP  Specific 

c.  ISP  Specific 

19.  Method  for  Collecting,  Validating,  and  Sharing  Data 

a.  SE  Specific 

b.  TEMP  Specific 

c.  ISP  Specific 

20.  Data/Information  Flow 

a.  TEMP  Deficiency  Reporting 

b.  Data  Quality  Requirements 

c.  System  Data  Exchange 

d.  Data  Timeliness 

e.  Information  Access 

21.  Integrated  Project  Schedule 

a.  Funding  Summary 

22.  Correlation  Matrix  Between  KPP/KSA,  MOE,  MOS,  CTPs 

a.  MOESandMOS 

b.  Traceability 

23.  Approach  for  Evaluating  System  Process  and  Maturity 

24.  Risks 

a.  System 

b.  Process 

25.  Critical  Technical  Parameters 

26.  Logistics 

a.  Resources 

b.  Special  Requirements 

27.  Sustainment 

28.  T&E  Strategy 

29.  Evaluation  Framework 

30.  Interoperability  Requirements 

a.  Supporting  Systems 

31.  Configuration  Differences  between  Current  System  and  System  to  be  Fielded 
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Appendix  B:  SEP  Orphan  Topics  and  Common  Themes 


Milestone 

Section 

Title 

A 

1.3 

Approach  for  SEP  Updates 

2.5 

Technology  Development  and  Evolving 
Acquisition  Strategy 

4.1 

Technology  Maturation  Responsibility 

5.1 

Event -Driven Technical  Reviews 

5.2 

Technical  Review  Management 

5.3 

Ch  a  i  ri  n  g  of  Te  ch  n  i  ca  1  R  e  v  i  e  ws 

5.4 

Stakeholder  Participation  in  Technical  Reviews 

5.5 

Peer  Participation  at  Technical  Reviews 

6.2 

Use  of  Critical  Paths  and  Technical  Reviews 

6.6 

Contracting  Considerations 

B 

1.3 

Approach  for  SEP  Updates 

2.2 

Statutory  and  Regulatory  Requirements 

4.1 

Technical  Baseline  Management  Responsibility 

4.4 

Specification  Tree  and  WBS  Link 

5.1 

Event  -Driven  Technical  Reviews 

5.2 

Technical  Review  Management 

5.3 

Chairing  of  Technical  Reviews  1 

5.4 

Stakeholder  Participation  in  Technical  Reviews  | 

5.5 

Peer  participation  at  Technical  Reviews 

6.2 

Program  Manager's  Approach  to  UsingTechnical 
Reviews 

6.6 

Contracting  Considerations 

C 

1.3 

Approach  for  SEP  Updates  ! 

2.2 

Comparison  of  Data  to  Planning  Assumptions 

2.4 

Production  and  Design  Driven  Operations  & 
Support  Costs 

3.1 

Lead/Chief  Systems  Engineer  and  Functional 
Leads 

4.1 

Technical  Baseline  Management  Responsibility 

4.4 

Technical  Baseline 

Common  Themes 

Milestone  and  Section 

SEP  Updates 

1.3  of  A.  B.  and  C 

Roles  and 

Responsibilities 

4.1  A  and  B  and  C 

Reviews 

5.1  -5.5A  and  B  and  C 

Contracting 

6.2  B  and  C 

6.6A  and  B  and  C 
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Appendix  C:  TEMP  Orphan  Topics 


Section 

Title 

Description 

1.1 

Purpose 

2.4 

TEMP  Updates 

3.3.1 

Mission-Oriented  Approach 

Evaluate  mission  performance  in  a  mission  context  (focuses  on  how  the  system  will  be  employed) 

3.3.2 

Developmental  Test  Objectives 

Summarize  the  planned  objectives  and  stat  the  methodology  to  test  the  system  attributes  defined  by 
the  appicable  capability  requirement  document 

3.3.4 

Test  Limitations 

3.4 

Live  Fire  Test  and  Evaluation  Approach 

3.4.1 

Live  Fire  Test  Objectives 

3.4.2 

Modeling  &  Simulation 

in  terms  of  life  fire 

3.4.3 

Test  Limitations 

3.6 

Operational  Evaluation  Approach 

Independent  evaluation  of  the  system 

3.6.3 

Test  Limitations 

3.7 

Other  Certifications 

3.3 

Reliability  growth 

4.1.1 

Test  Articles 

Actual  number  and  timing 

4.1.2 

Test  Sites  and  Instrumentation 

4.1.3 

Test  Support  Equipment 

4.1.4 

Threat  Representation 

4.1.5 

Test  Targets  and  Expendables 

4.1.6 

Operational  Force  Test  Support 

4.1.7 

Models,  Simulations,  and  Testbeds 

4.1.3 

Joint  Mission  Environment 

Live,  virtual,  or  constructive  components  for  an  acceptable  environment 

4.2 

Federal,  State,  and  Local  Requirements 

environmental  regs 

Appendix  D:  ISP  Orphan  Topics 


Chapter  1:  Introduction 

Project  Info 

2.3  Step  3:  Determine  the  operational  users  and 
notional  suppliers  of  the  information  needed. 

2.9  Step  9:  Discuss  RF  Spectrum  needs. 

2.10  Step  10:  Perform  a  Net-Centric  Assessment 

2.12  Step  12:  Discuss  the  program's  Information 
Assurance  strategy  and  reference  the  Program 
Protection  Plan. 

IAS 

2.13  Step  13:  Identify  information  support  needs 
to  support  development,  testing  and  training. 

Appendix  D.  -  Acronym  List 

ISP 
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Appendix  E:  ISP  Example  Orphan  Topics 


(U)  EXECUTIVE  SUMMARY 

1 

(U)  INTRODUCTION 

1.1.2 

(U)  Relationship  to  Other  Programs 

1.1.3 

(U)  Relationship  to  Relevant  Joint 

Functional  Concepts  fJFCs).  Joint 

1. 1.3.1 

(U)  Joint  Functional  Concepts 

1.1.3. 2 

(U)  Associated  Integrated  Architectures 

1.1.3. 3 

(U)  JCIDS 

1.2 

(U)  PROGRAM  DATA 

Current  MS  and  Acquistion  Status 
Integrated  Master  Schedule 

Increment  1  schedule 

Increment  II  schedule 

1.2.1 

(U)  Milestone  and  Acquisition  Status 

1.2.2 

{ U )  S  p  i  ra  1  Ev  o  1  ut  ion  Strategy 

1.2.3 

(U)  Program  Points  of  Contact 

1.3.1 

(U)  Information  Integrity 

1.3.2 

(U)  DoD  PKI  System  Architecture 

1.3. 2.1 

(U)  DoD  PKI  Certificate  Management 
Components 

1.3.3 

(U)  Role  Definitions 

1.3.4 

(U)  PKI  System  Interface  Overview 

1.4 

(U)  ISP  DOCUMENT  STRUCTURE 

2 

(U)  ANALYSIS 

2.3 

(U)  STEP  3  -  DETERMINE  OPERATIONAL 

USERS  AND  NOTIONAL  SUPPLIERS 

OV-4  Organizational  Relationship 

Role  Overview 

2.3.1 

(U)  Operational  Nodes  and  Elements  (OV-2) 

Operational  Nodes  and  Elements 
(OV-2) 

2.3.2 

(U)  Operational  Node  Activities 

Operational  Node  Activities  (SV-5) 

2.12.1 

(U)  Program  Category  and  Life -Cycle  Status 

2.12.2 

(U)  Mission  Assurance  Category  and  Confidentiality 
Leve  1 

2.12.3 

(U)  System  Description 

2.12.4 

(U)  Thre  at/Risk  Assessment 

2.12.5 

(U)  IA  Re qui re ments 

2.12.6 

(U)  Certif  ication  and  Accre  ditati  on 

2.12.7 

(U)  IA  Testing 

2.12.3 

(U)  1 A  Analysis 

2.13 

(U)  STEP  13:  IDENTIFY  SUPPORT  NEEDS  FOR 

D EV ELO PMENT.  TESTI  N  G .  AND  TRAININ  G 

2.13.1 

(U)  Devel  opme  nt 

2.13.2 

(U)  Te  sti  ng 

2.13.3 

(U)  Developmental  Test  and  Evaluation  (DT&E) 

2.13.4 

(U)  Operational  Test  and  Evaluation  (OT&E) 

2.13.5 

( U )  Tra  i  n  i  n  g 

2.13.6 

(U)  CC/S/A  Training  Requirements 

2.13.7 

(U)  LRA/TRA  Background,  quallif icati ons,  experience, 
and  clearance  requirements 

3 

(U) ISSUES 

Appendix  A 

Ref  e  re  rice  s 

Appendix  D 

Acronym  List  and  Glossary  (AV-2) 

Appendix  E 

Publ  i  c  Key  1  nf  ra  structure  Overview  and  Summary 
Information  (AV-1) 

Appe ndi x  F 

Key  Interface  Profile  (KIP) 

Appendix  G 

Data  AND  Service  Exposure 

2.5 

(U)  STEP  9  -  DISCUSS  RADIO  FREQUENCY  SPECTRUM 

NEEDS 

2.1 

(U)  STEP  10  -  PERFORM  A  NET-CENTRIC  ASSESSMENT 

2.10.1 

(U)  Step  10-A:  Evaluate  Program  Agai nst 

Measurement  Criteria 

2.10.1.1 

(U)  PKI's  Incorporation  of  NCOW  RM  Capabilities  and 
Se  rvice  s 

2.10.1.2 

(U)  Techni  cal  View  Products 

2.10.1.3 

(U)  SV-TV  Bridge 

2.10.1.4 

(U)  Definitions  and  Vocabulary 

2.10.1.5 

( U )  GIG  Mission  Area  Initial  Capabilities  Document 
(MA  1  CD) 

2.10.2 

(U)  Step  10-B:  Compliance  with  Emerging  NCES  CESs 

2.10.3 

(U)  Step  10-C:  Assess  the  Use  of  Software-Compliant 

Radi  os 

2.10.4 

(U)  Step  10-D:  Assess  the  Use  of  IPv6  DoD  Net- 
Centric  Data  Strategy 

2.10.5 

(U)  Step  lOe:  Assess  the  Use  of  DoD-Ce ntric  Data 

M  ana  ge  m  e  nt  St  rat  e  gy 

2.10.6 

(U)  Step  10-F:  Assessthe  GIG  Bandwi dth  Expansion 

Rel  ationshi  p 

2.10.7 

(U)  Step  10-G  Net-Ready  Key  Performance 

P  a  ra  meter  ( N  R  -K  P  P )  State  m  e  nt 

2.10.3 

(U)  Appl  i  cabiility  of  Maj  or  N  et-Centricity 

Characte  rf sties  of  PKI  Increments  One  and  Two 

2.12 

(U)  STEP  12:  DISCUSS  THE  INFORMATION  ASSURANCE 

STRATEGY 
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